
Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



© Publication number: 



0 401 077 

A2 



EUROPEAN PATENT APPLICATION 



© Application number: 90401304.2 
(§) Date of filing: 16.05.90 



© int. ci. 5 : G06F 15/68, H04N 1/40 



The title of the invention has been amended 
(Guidelines for Examination in the EPO, A-lll, 

7.3). 


© Applicant: BIOLOGICAL VISIONS 
1510 Fashion Island Boulevard 
San Mateo, California(US) 


© Priority: 18.05.89 US 354255 

© Date of publication of application: 
05.12.90 Bulletin 90/49 


@ Inventor: Rutherford, Hal 
4143 Middlepark Drive 
San Jose, California(US) 


© Designated Contracting States: 
CH DE FR GB IT LI NL SE 


© Representative: Kopacz, William James 
83, Avenue Foch 
F-75116 Paris(FR) 


© Method and apparatus for removing noise data from a digitized image. 


© An apparatus and method for removing back- 
ground noise and high frequency noise form an 
image by comparing each pixel in the image with 
neighboring pixels defining a variably shaped and 


sized kernel. The size and shape of the kernel are 
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METHOD AND APPARATUS FOR GENERATING QUANTIFIABLE VIDEO DISPLAYS 



Background of the Invention 

The invention pertains generally to the field of 
image processing, and, more particularly, to the 
field of generating quantifiable images from digital 
images representing data spatially as pixel patterns 
of greater and lesser intensity. 

The preferred embodiment of the invention is 
adapted for analysis of biological data generated in 
recombinant DNA research and other biological 
research. Such data includes 2D gels, DNA 
sequencing gels, gel blots, RFLP, DNA blots, 
microtiter color, microtiter fluorescence and other 
types of data presented spatially in an image. 
Typically, such images consist of a plurality of 
pixels with areas of pixels of varying intensity re- 
presenting some amount of a particular DNA or 
protein with the intensity attributable to the protein 
being superimposed upon intensity representing 
background noise and high frequency noise caused 
by such things as pinholes in the film, penetration 
of the film by gamma rays etc. 

Although the invention will be described in 
terms of its application to biological data, it will be 
appreciated that the teachings of the invention have 
utility in other fields of analysis of images. 

A problem in analyzing such data in the past 
so as to be able to quantify the amount of a protein 
represented by a particular area of pixels in the 
image has been how to separate the intensity re- 
presenting the data from the intensity caused by 
background noise. Although pixel intensity is the 
concept used herein to convey the teachings of the 
invention, pixel value is the general concept con- 
templated by the teachings of the invention. That 
is, the pixel values being analyzed may represent 
. something other than light intensity. For example, 
each pixel in an image may represent the strength 
of radio transmissions from a small sector of the 
sky such that the invention could be used in radio 
astronomy applications. 

In the past, such techniques as rolling ball 
filters have been used for background noise re- 
moval from images. Such a teaching is found in a 
conference paper by Rutherford et al. entitled 
"Object Identification and Measurement from Im- 
ages with Access to the Database to Select Spe- 
cific Subpopulations of Special Interest" published 
at the E-0 Lase and E-0 Imaging Conference 
sponsored by S.P.I.E., January 1987 with the pro- 
ceedings published in May of 1987. There, the 
authors describe a method of background correc- 
tion, i.e., noise removal, by use of a rolling ball 
filter which effectively takes the minimum pixel 
value in the ball filter region as the pixel value for 



the background image. The resultant image is then 
subtracted from the digitized image. A pipeline 
image array processor is used to perform this 
process. Such a technique however is not op- 

5 timized for removal of background noise and high 
frequency noise in all situations because it does 
not take into account the varying geometric shapes 
of the data of interest in many varied application 
and because it does not take into account other 

10 application specific phenomenon such as vertical 
noise strips, dead spaces etc. 

Accordingly, a need has arisen for apparatus 
and a method to optimize the noise removal pro- 
cess for data presented in many varied spatial 

75 formats. 



Summary of the Invention 

20 According to the teachings of the invention, 

there is disclosed herein a method and apparatus 
for background noise removal which uses a vari- 
able shape and a variable size kernel or neighbor- 
hood of adjacent pixels surrounding or next to the 

25 pixel being processed. The value of the pixel being 
processed is compared to all, or some selected 
subset, of the pixels in the neighborhood to find the 
minimum value. This minimum value is then substi- 
tuted for the value of the pixel being processed. 

30 When all pixels have been so processed by com- 
paring them to the values of the surrounding pixels 
in the corresponding neighborhood (each pixel has 
its own neighborhood), the resulting image is a 
"background image". A background image is an 

35 image where each pixel has the value of the small- 
est valued pixel in the neighborhood to which it 
was compared. Of course, those skilled in the art 
will appreciate that the background removal pro- 
cess can also be performed on a reverse video 

40 image by finding the maximum pixel value in each 
neighborhood and substituting that value for the 
value of the pixel of interest corresponding to that 
neighborhood. 

The background image may then be further 

45 processed in some embodiments to remove high 
frequency noise. In one embodiment, high frequen- 
cy noise is removed by processing the background 
image to generate a "maximum image", i.e., an 
image generated from the background image 

so showing the maximum pixel values for each neigh- 
borhood in the background image. This maximum 
image is generated by using a smaller neighbor- 
hood than used in generating the background im- 
age and then using this smaller neighborhood to 
process the background image as follows. Each 
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pixel has its value compared to the values of the 
pixels in a corresponding neighborhood of sur- 
rounding pixels. The maximum value in each 
neighborhood is then substituted for the value of 
the corresponding pixel. When this has been done 
for all pixels, the maximum image is complete. This 
maximum image is substantially devoid of high 
frequency, large amplitude noise which dips below 
the surrounding neighborhood such as is char- 
acteristic of pinhole defects in film etc. This maxi- 
mum image is subtracted from the starting image 
to generated a "background removed image". 

In another embodiment according to the teach- 
ings of the invention, the maximum image is used 
as a starting image in an apparatus to perform a 
process to remove high frequency, low amplitude 
noise. In this process, a neighborhood is used 
which is smaller than the neighborhood used to 
generate the background image. Each pixel value 
for a pixel of interest is added to the pixel values 
for all the other pixels in the corresponding neigh- 
borhood. The sum is then divided by the number 
of pixels in the neighborhood to derive an average 
pixel value, and the value of the pixel of interest is 
set equal to this average value. After this is done 
for all pixels, the resultant image is subtracted from 
the original image used to generate the back- 
ground image to arrive at a background removed 
image. 

In another alternative embodiment, the average 
image may be generated directly from the back- 
ground image and the resulting image subtracted 
from the original image to derive the background 
removed image. 

In another alternative embodiment, the image 
generated by the averaging process may be gen- 
erated directly from the background image, and the 
resulting image is used as the input image for the 
process of generating the "maximum" image. The 
resulting image is subtracted from the original im- 
age to derive the background removed image. 

Brief Description of the Drawings 



Figures 1(a) and 1(b) show, respectively, a 
typical autoradiograph of a 1-D gel separation and 
the same image with the data removed leaving 
only the background intensity showing. 

Figure 2 a drawing showing how the data 
bearing image pixel value profile compared to the 
background pixel value profile. 

Figure 3 is a block diagram of the hardware 
which can be used according to the teachings of 
the invention. . 

Figure 4 is a flow chart for the process for 
removing background noise from the image. 

Figures 5 through 7 illustrate the process of 



background noise removal by comparison to neigh- 
borhood pixel values. 

Figure 8 is a more detailed flow chart of the 
background noise removal process, 
s Figures 9 and 10 are alternative processes 

for selection of kernel size and shape. 

Figures 11 and 12 illustrate how different 
kernel shapes are optimized for various data ap- 
plications. 

w Figure 13 is a flow chart for the preferred 

embodiment of the process of background noise 
removal. 

Figure 14 is a more detailed flow chart illus- 
trating the process of generating a maximum im- 
75 age. 

Figure 15 is a more detailed flowchart illus- 
trating the process of high-frequency, low-ampli- 
tude noise removal by averaging. 

Figure 16 is a flow chart for the process of 
20 generating a percent change image. 

Figures 17(a) through 17(e) are the compo- 
nents of a quad display and the quad display itself. 

Figure 1 8 is another type of quad display. 

Figure 19 illustrates the concept of linked 
25 cursors for the quad display. 

Figure 20 illustrates the process for align- 
ment of images 1 and 2 which must be performed 
prior to the computation of values for the cursor 
locations in the quad display. 

30 

Detailed Description of the Preferred Embodiment 



Referring to Figure 1(a), there is shown an 
image of a typical autoradiograph of a 1-D protein 

35 separation. Each of lanes 10 and 24 contains sepa- 
rated bands of radioactively labeled proteins from 
different samples. For example, lane 10 contains 
bands 12 and 14 with the difference in pitch of the 
crosshatching of band 14 indicating that this band 

40 is of greater brightness or intensity than the inten- 
sity of band 12. Likewise, bands 16, 18, 20 and 22 
in Figure 1(a) all have varying degrees of bright- 
ness or intensity. A similar situation exists for lane 
24, which is separated from lane 10 by a dead 

45 space 26. The varying intensity of each band is 
indicative of the amount of the particular protein or 
proteins represented by that band which was 
present on the gel at that particular position. 

It is useful to be able to quantify an image 

so such as shown in Figure 1 (a) such that the intensity 
of the various bands can be measured as an in- 
dication of the amount of protein represented there- 
by. The difficulty with this approach, however, is 
that the various bands have their intensities super- 

55 imposed upon background noise which, because of 
its varying intensity across a lane, causes errors. 
That is, the background noise can be thought of as 
forming an image of varying intensity which would 
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still be present even if there were no data repre- 
sented in Figure 1(a). Figure 1(b) is a drawing 
showing this background image. The background 
image has a lane 10 which corresponds to iane 10 
in Figure 1 (a) and a lane 24 which corresponds to 
the lane 24 in Figure 1 (a). The differences in pitch 
in the crosshatching of lanes 10 and 24 conveys 
in pictorial form the variation in the intensity or 
brightness of the background noise in the lane at 
various locations. For example, the area 28 in iane 
10 has a brighter background intensity than the 
area 30. By superimposing the image of Figure 1- 
(a) on the image of Figure 1(b), it can be seen that 
the relatively brighter intensity of band 14, which 
overlies an area of lesser intensity in the back- 
ground image, as compared to a less-bright band 
18, which overlies an area of brighter background 
intensity in Figure 1(b). For this reason, the relative 
intensities of the bands 14 and 18 cannot be used 
directly to quantify the amount of protein at those 
respective positions in the gel without creating er- 
rors caused by the varying intensity of the back- 
ground image along lane 10'. Thus, according to 
the teachings of the invention, the background im- 
age of Figure 1(b) is derived by image processing 
of the image represented by Figure 1(a), and the 
resulting image is then subtracted from the image 
of Figure 1 (a) to leave a quantifiable data image. 

Referring to Figure 2, there are shown com- 
parative intensity profiles through the image of Fig- 
ure 1 (a) to show the effect of background removal. 
The intensity trace labeled 32 represents the inten- 
sity of the original image which includes both inten- 
sity attributable to data as well as intensity attrib- 
utable to background noise. The trace labeled 34 
represent the intensity of the original image after 
background removal and, therefore, represents the 
intensity attributable to the quantity of a particular 
protein located at the corresponding location on the 
gel. 

According to the teachings of the invention, a 
manually manipulated cursor having a variable size 
and a variable shape may be placed over any band 
of interest in the background-removed image to 
determine the intensity of that band attributable to 
the presence of a protein of interest. The process 
of determining the intensity caused by the data 
essentially involves the process of integrating the 
trace 34 to determine the area under any particular 
peak. Typicaiiy, the result of this integration will be 
reported at the touch of a key on a computer 
keyboard. 

Referring to Figure 3, there is shown a block 
diagram of a computer apparatus according to the 
teachings of the invention. Image film 36, which 
contains a spatial depiction of the data to be ana- 
lyzed, is placed on a light box 38. The light box 
shines light through the image film to create a 



pattern of light which has varying spatial intensity 
in accordance with the data and the background 
noise. Also, the image may be acquired by shining 
light on a nontranslucent film such as a polaroid 

5 shot. The resulting light pattern contains the data to 
be analyzed. This light pattern is converted by a 
video camera 40 into a video signal on line 42 
representing an analog form of a raster-scanned 
version of the image on film 36. This analog signal 

w is digitized in a data converter interface 44 and 
results in a stream of digital data on bus 46. This 
stream of digital data is read by a computer 48 and 
is stored in memory for further image-processing 
operations. The computer 48 is typically an IBM 

75 ATTM personal computer with an image processor 
card set plugged into the card slots. The image 
processor card set is an off-the-shelf, image-pro- 
cessing circuit manufactured by Matrox under the 
trademark MVP-ATTM Image-Processing Card Set. 

20 The computer 48 interfaces with the user through a 
mouse 50, a keyboard 52 and a monitor 54. An 
external memory 56 stores data and programs. 
Several software modules according to the teach- 
ings of the invention are shown as stored in mem- 

25 ory 56. They are: a neighborhood shape/size inter- 
facing module 58; a background removal module 
60; a smoothing module 62; and an averaging 
module 64. The neighborhood shape/size interface 
module 58 serves to determine the shape and size 

30 of a neighborhood or kernel of pixels the values of 
which will be compared to the value of a pixel of 
interest in the kernel to determine the spatial inten- 
sity patterns of the background image. Typically, 
the shape of the neighborhood is determined by 

35 the computer for the particular application involved 
and relates to the typical shape of the data patterns 
to be analyzed. However, in alternative embodi- 
ments, the shape of the neighborhood may be set 
by the user in any of several different ways. For 

40 example, at start-up time, or upon switching ap- 
plications, the computer can prompt the user 
through monitor 54 to determine what type of data 
is to be analyzed. After the user responds, either 
through the keyboard 52 or the mouse 50, the 

45 computer can put up either a textual, verbal or a 
pictorial menu of neighborhood shapes to be used. 
The user can then indicate which shape to use 
either by selecting it with mouse 50 or by typing in 
the code for the shape via keyboard 52 or by 

50 stating the shape. Alternatively, the user may 
sketch the neighborhood shape and/or size to be 
used through use of the mouse 50. In some em- 
bodiments, the shape of the neighborhood will be 
selected by the computer 48 based upon the user 

55 response regarding what type of data is to be 
analyzed. In some embodiments, a first neighbor- 
hood shape will be used to get rid of particular 
noise patterns having specific shapes followed by 
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the use of anther shape for the neighborhood which 
is keyed to the shape of the particular data or 
application for which the teachings of the invention 
will be used. 

The size of the neighborhood to be used gen- 
erally depends upon the typical size of the data 
spatial patterns to be analyzed. In the preferred 
embodiment, the size of the neighborhood is cho- 
sen which has a largest dimension which is two 
and one-half times the size of the largest data 
spatial pattern to be analyzed. In the preferred 
embodiment, the user may be prompted for the 
desired size for the neighborhood and may re- 
spond either in terms of a number or a code for the 
desired size. Alternatively, the selected shape for 
the kernel may be displayed on the screen, and 
the user may adjust the size of the kernel by 
having the kernel superimposed upon the image to 
be analyzed and using a "rubber band "-type cur- 
sor to adjust the size of the kernel. The details as 
to how the shape and size of the kernel to be used 
are selected by the user or by the computer are 
not critical to the invention. 

The details of the remove background image 
module 60 will be described in greater detail be- 
low. The basic function of this module is to deter- 
mine the level of background intensity throughout 
the image to be analyzed and to create a back- 
ground image reflecting that background intensity 
at all points in the background image. This back- 
ground image may then be subtracted from the 
original image in some embodiments to derive a 
background removed image. 

The smooth background image module 62 re- 
moves high-frequency, high-amplitude noise (high- 
amplitude noise for purposes of this invention 
means noise which dips below the level the sur- 
rounding neighborhood) by finding the maximum 
pixel in each kernel of the background image and 
setting the value of the pixel of interest in this 
kernel to the maximum value found in the kernel. 
When this is done for all pixels and their cor- 
responding kernels "maximum" background image 
has been completed. This serves to get rid of high 
frequency, large amplitude noise characterized by 
pixels of low intensity in the background image 
such as might be caused by pinholes in the film, 
gamma rays, etc. 

Finally, the averaging module 64 gets rid of 
high frequency, small amplitude noise in either the 
background image or the smoothed background 
image generated by module 62. This is done by 
averaging all the pixels in a neighborhood and 
setting the pixel of interest in each neighborhood to 
the average value. Both modules 62 and 64 will be 
described in more detail below. 

Referring to Figure '4, there is shown a flow 
chart for a basic embodiment of a process accord- 



ing to the teachings of the invention for background 
removal. The first step, symbolized by block 66, is 
to acquire the image to be analyzed. Specifically, 
the image to be analyzed is digitized into a plural- 

5 ity of pixels. These pixels define an image which 
contains data to be analyzed and displays this data 
in terms of varying spatial patterns of intensity, 
color, fill pattern or other means of displaying val- 
ues for the pixels. How the value for each pixel is 

70 depicted is not critical to the invention. Typically, 
pixel values will be displayed in terms of their 
intensities. For some applications, the data to be 
analyzed is shown as dark spots on a lighter back- 
ground such as autoradiography. In those applica- 

75 tions, a "negative" or reverse video image is gen- 
erated from the acquired image before further pro- 
cessing. In other applications, the data to be ana- 
lyzed is shown as lighter spots on a dark back- 
ground. In such applications, the acquired image is 

20 used as is without doing a reverse video image. In 
some embodiments, it is useful to average the 
original acquired image before further processing 
to remove the background. This averaging process 
is identical to the process described below with 

25 reference to Figure 15 carried out on the back- 
ground removed image or the image generated by 
the process described with reference to Figure 14. 

Next, the computer system interacts with the 
user to select a particular kernel size and/or shape 

30 for use in generating the background image, as 
symbolized by block 68. As noted earlier, the ker- 
nel shape is typically selected by the computer 
based upon the type of data to be analyzed in the 
preferred embodiment. That is, if the data takes the 

35 form of vertical rectangular blocks, as in the case 
of one-dimensional separations of DNA or proteins, 
then the preferred kernel shape is usually rectan- 
gular. However, if the data to be analyzed takes the 
form of circular spots such as in DNA library 

40 screens, cells tagged with fluorescing antibodies, 
or images of 96-well microtiter plates, then the 
preferred kernel shape is circular. 

Generally speaking, the size of the kernel 
should be substantially larger than the size of the 

45 largest data area to be analyzed. That is, if the 
largest data spot to be analyzed is a circle of 2 mm 
diameter, then the preferred kernel shape and size 
is a circular area having a diameter sufficient to 
cause the total are within the kernel to be approxi- 

50 mately 2.5 times the radius of the 2 mm diameter 
data spot. The reason for this size relationship is to 
insure that at least some background area outside 
the area of data of interest are included within the 
kernel. This is necessary to insure that a proper 

55 background image is generated. This is because 
the process of generating the background image 
involves comparing the value of each pixel in the 
image to be analyzed to the values of the sur- 
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rounding pixels to find a minimum value char- 
acteristic of the background. Thus, if no back- 
ground pixels are included within a kernel which 
happens to be centered over a data spot, then the 
minimum intensity value which will be found in that 
kernel will not in fact be representative of back- 
ground intensity at that location but will be repre- 
sentative of the intensity of the data as superim- 
posed upon the intensity of the background. 

As noted earlier herein, the kernel size may be 
selected by the user using any one of a number of 
different methods, none of which are critical to the 
teachings of the invention. Alternatively, the kernel 
size may be selected by the computer automati- 
cally, based on the type of data being analyzed. In 
the preferred embodiment, the kernel shape is se- 
lected by the computer automatically, based upon 
the data being analyzed, and the kernel size is 
selected by the user using a "rubber band" cursor 
to adjust the size of a default kernel which is 
superimposed over the image of data to be ana- 
lyzed. The user then touches an edge of the kernel 
and "drags" it out to an appropriate dimension in 
some embodiments. In the case of rectangular 
kernels, the user may touch each of two opposing 
sides and drag each one individually out to the 
appropriate dimension so as to obtain the desired 
size and aspect ratio. This is done after dragging 
the kernel to a desired position on the image to be 
analyzed so as to surround the largest area of data 
shown on the image. 

Block 70 represents the process of actually 
generating the background image using the kernel 
selected by the user. This process is best under- 
stood by reference to Figure 5, which shows a 
typical kernel or neighborhood 72 of rectangular 
shape surrounding a pixel of interest 74. Figure 6 
shows the relationship of the kernel 72 to the 
overall image being processed. The pixel of inter- 
est 74 is any pixel within the area encompassed by 
the kernel 72. Although typically the pixel of inter- 
est is in the center of the kernel in the preferred 
embodiment, in alternative embodiments the pixel 
of interest 74 may be located anywhere within the 
boundaries of the kernel 72. The pixel of interest 
74 is shown in the middle of a raster scan line 76 
and is in the middle of a column of pixels 78. As is 
best seen in Figure 6, the pixel of interest 74 is a 
single pixel in a line of pixels which together com- 
prise the single raster scan line 76 of the image to 
be analyzed 80. The image 80 is comprised of 512 
raster scan lines like the raster scan line 76 in 
some embodiments, and there are typically 512 
pixels on each raster scan line. The size of the 
raster is not critical to the invention. The kernel 72 
includes several pixels from the raster scan line 76 
within its boundaries and includes several other 
raster scan lines both above and below the raster 



scan line 76 although these other raster scan lines 
are not shown in Figure 6 to avoid unnecessary 
complexity. 

Referring again to Figure 5, the process of 

5 generating the background image is accomplished 
by comparing the value of the pixel 74 to the 
values of each of the other pixels in the kernel 72 
and finding the minimum value pixel and substitut- 
ing its value for the current value of the pixel 74. 

ro For example, assume that the pixel 74 has a value 
of 5 on a scale from 1 to 1 0. Assume also that the 
pixels 82, 84 and 86 in the raster scan line 88 have 
values of 7, 4 and 1 , respectively. When the value 
of the pixel 74 is compared to the value of the pixel 

15 82, the value of the pixel 74 will be less, and no 
substitution is made. When the value of the pixel 
74 is compared to the value of the pixel 84, it will 
be found that the value of the pixel 84 is less than 
that of pixel 74, and a substitution will be made 

20 such that the value of pixel 74 is rewritten to be a 
4. When pixel 74 is compared to pixel 86, it will be 
found that pixel 86 has a still smaller value of 1, 
and this value of 1 will be written to pixel 74. 

This process continues until all the other pixels 

25 in the kernel 72 have been examined. Each time a 
new minimum is found, that value is used to up- 
date the value of the pixel 74. When this compari- 
son process is completed for every pixel in the 
kernel 72, the final value of the pixel 74 will be 

30 established for use as one pixel in the background 
image. This process of comparing each pixel in the 
image 80 of Figure 6 to all the pixels in a kernel 
comprised of a plurality of pixels adjacent to the 
pixel of interest is repeated for every pixel in the 

35 512 by 512 pixel array of the image 80. When it 
has been completed, the complete background im- 
age has been generated. 

The process symbolized by block 70 contem- 
plates simultaneous processing for each pixel in 

40 the image 80 such that each pixel in the image is 
compared simultaneously with one other pixel in a 
kernel of pixels adjacent to the pixel of interest, 
and this process is repeated simultaneously for all 
pixels until all the pixels of interest have been 

45 compared to all the pixels and their respective 
kernels. This substantially increases the speed of 
processing to generate the background image. In 
some alternative embodiments, only a selected 
subset of the other pixels in each kernel will be 

so sampled. In still other alternative embodiments, the 
process of comparing each pixel with the adjacent 
pixels in its kernel may be done serially such that 
each pixel of interest is compared simultaneously 
with all or some subset of all the pixels in the 

55 corresponding kernel such that the entire kernel is 
searched in a single machine cycle or however 
many machine cycles are necessary to make the 
comparison between one pixel and another. After 
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this process is accomplished, another pixel of inter- 
est from the image to be analyzed is selected and 
simultaneous comparison is made of this pixel with 
al! or some subset of the pixels in the kernel 
corresponding to that pixel. 

Note that as the pixel of interest moves along a 
raster line, a corresponding kernel surrounding that 
pixel of interest is selected to keep the pixel of 
interest in the same relative location within the 
boundaries of the kernel. 

Note also that the background image genera- 
tion process must be performed using a copy of 
the image such that the updating of the values of 
each pixel of interest occurs in the copy. This is 
necessary because each pixel of interest is a 
neighboring pixel for the kernel corresponding to 
some other pixel of interest. Therefore, if the value 
of the pixel of interest in the image to be analyzed 
is updated prior to having processed all the other 
pixels in the image, there will be distortions and 
errors caused in the processing of other pixels 
whose kernels overlap the pixel which had its value 
changed. 

After the background image is generated, it is 
subtracted on a pixel-by-pixel basis from the image 
to be analyzed, as symbolized by block 90 in 
Figure 4. That is, the value of pixel 1 in raster scan 
line 1 of the background image is subtracted from 
the value of pixel 1 in raster scan line 1 of the 
image to be analyzed. 

After this process is completed, the resultant 
image is displayed as a background-removed im- 
age on the monitor 54 in Figure 3, as symbolized 
by block 92 in Figure 4. 

Referring to Figure 7, there is shown symboli- 
cally the process by which whole image processing 
occurs in the computer apparatus according to the 
teachings of the invention. Simultaneous compari- 
sons of each pixel in the image to be analyzed 80 
to a single one of the adjacent pixels in the cor- 
responding kernel is accomplished by the use of 
offset and compare commands to the image pro- 
cessing board set in the IBM ATTM. For example, 
assume that the image to be analyzed 80 is com- 
prised of 9 pixels labeled A through I. Assume also 
that the heavy line 94 defines the boundaries of a 
kernel for a pixel of interest E. The phrase "pixel of 
interest" as the phrase is used herein means the 
pixel being processed which has its value com- 
pared to the other pixel values in the kernel or 
neighborhood and which has its value replaced if 
the test of the comparison is satisfied, i.e., in the 
case of generation of the background image, if the 
neighboring pixel selected from the other pixels in 
the kernel has a value which is less than the value 
of the pixel of interest. 

To further the illustration, assume also that the 
heavy line 96 defines the boundaries for a kernel 



for the pixel I. Similarly, a kernel comprised of the 
pixels D, E, G and H can be defined for the pixel 
H, and a kernel comprised of the pixels B, C, E 
and F can be defined for the pixel F. 

5 Now assume that the first comparison in the 

process of generating the background image is to 
compare the values of the pixels of interest in all 
these kernels, i.e., the pixels at the lower right-hand 
corner of each kernel, to the values of the pixels at 

10 the upper left-hand corner of each kernel. Thus, in 
the case of kernel 94, the value of pixel E, the pixel 
of interest, is compared to the value of the pixel A. 
If the value of A is less than the value of E, then 
the value of A will be substituted for the value of E 

15 in a copy of the image 80. This copy is shown to 
the right and is labeled the "offset" image. Simulta- 
neously, the value of pixel I is compared to the 
value of the pixel E. If the value of E is less than 
the value of I, then the value of I will be overwritten 

20 with the value of E in the offset image. 

The offset image 98 is originally a copy of the 
image to be analyzed 80. To facilitate simultaneous 
comparison of some or all of the pixels in image 80 
to one of the pixels in their corresponding kernels, 

25 the offset image 98 is used as follows. Imagine the 
offset image 98 is a transparency which can be 
placed over the image 80 and shifted about so as 
to align any pixel with any other pixel. For the first 
comparison in the hypothetical example, the pixel 

30 E will be compared with the pixel A. To implement 
this, the memory map of digital data representing 
the "transparency", i.e., the offset image 98, is 
electronically placed over the memory map of digi- 
tal data representing the original or "acquired" 

35 image 80 such that the offset image pixel A lies on 
top of the pixel E and the offset image pixel B lies 
on top of the pixel F in image 80. This aligns the 
offset image with the image 80 such that each pixel 
in the image 80 which has a pixel in the offset 

40 image 98 overlying it will be aligned with the pixel 
to which it is to be compared for the first round of 
comparisons. That is, the pixel E will be aligned 
with the pixel A and the pixel F will be aligned with 
the pixel b'. Likewise, the pixel H will be aligned 

45 with the pixel d' and the pixel l will be aligned with 
the pixel e'. Examination of the kernels of image 
80 indicates that for each of the overlapped pixels 
in image 80, i.e., the pixels of interest, the over- 
lying pixel will be the pixel in the upper left-hand 

so corner of the kernel in image 80 which corresponds 
to each pixel in image 80 which is overlapped. The 
offset for this first round of comparisons, then is "1 
pixel up, 1 pixel left". 

The value of each pixel in image 80 which is 

55 overlapped is then compared to the value of the 
pixel which overlaps it in image 98. If any of the 
pixel values in image 98 are less than the pixel 
values in pixel 80 which they overlie, the minimum 
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value is used to update the pixel in the offset 
image corresponding to the pixel in the image 80. 
The pixels in the offset image 98 which correspond 
to the pixels in the image 80 are those with the 
same "relative address". To aid in understanding 
the meaning of the phrase "relative address" one 
can think of the labels A, B, etc. for the pixels in 
image 80 as their relative addresses or labels in 
memory. Thus, if the value of the pixel A is less 
than the value of the pixel E, then the value of the 
pixel a' is written into the memory location storing 
the value of the pixel E . The same process occurs 
for all other overlapped pixels. 

Pixels in image 80 which are not overlapped, 
such as the pixels A, B, C, D and G, are compared 
to "dummy" pixels (constants or locations in the 
memory map which are loaded with constants) in 
the offset image 98 which have had their values 
artificially set to the maximum intensity level. This 
has the effect of causing the non-overlapped pixels 
in image 80 to never have their corresponding 
pixels in image 98 replaced with a minimum. The 
dummy pixels are labeled with Xs in image 98. 

Thus, after one round of comparisons, each 
pixel in the image 80 will have been compared to 
one of the pixels in the corresponding kernel or a 
dummy pixel. To complete the process of generat- 
ing the background image, a new offset or shift is 
performed to align the offset image with the image 
80 such that the next pixel to be examined in each 
kernel overlies the pixel of interest in each kernel. 
Thus, for example, if the pixel E is to be compared 
next with the pixel D, then the offset image 98 is 
shifted such that the pixel d' overlaps the pixel E, 
and the pixel E overlaps the pixel F. Then a new 
round of comparisons is made simultaneously to 
compare all overlapped pixels with the values of 
their overlapping pixels with appropriate updating 
where new minimums are found. 

This offset/compare/update process occurs as 
many times as there are pixels in each kernel to be 
compared with the pixel of interest in each kernel. 
It is not necessary that all neighboring pixels in 
every kernel be compared with the pixel of interest 
to generate the background image. In fact, in some 
embodiments, only a sampling of the other pixels 
in each kernel is used to generate the background 
image. 

The above-described process is graphically il- 
lustrated in the flow chart of Figure 8. The process 
illustrated in Figure 8 corresponds to the process 
symbolized by block 70 in Figure 4. No further 
discussion of Figure 8 is deemed necessary since 
it is self explanatory in light of the discussion 
above. Of course, those skilled in the art will appre- 
ciate that the background removal process can also 
be performed on a reverse video image by finding 
the maximum pixel value in each neighborhood 



and substituting that value for the value of the pixel 
of interest corresponding to that neighborhood. 
This technique is deemed sufficiently self-explana- 
tory as to not warrant further discussion. 

5 Referring to Figure 9, there is shown a flow 

chart for the preferred embodiment of the process 
symbolized by block 68 in Figure 4. Basically, the 
process represented by the flow chart of Figure 9 
represents an embodiment where the computer 

10 prompts the user to indicate what type of data is 
being analyzed and then selects the appropriate 
kernel shape based upon the user response. The 
user is then prompted to select an appropriate size 
for the kernel given the shape selected by the 

75 computer. 

Referring to Figure 10, there is shown an alter- 
native embodiment of the process symbolized by 
block 68 in Figure 4. The difference between the 
embodiment shown in Figure 9 and Figure 10 is 

20 that in the embodiment of Figure 9, the computer 
selects the kernel shape based upon the user- 
supplied data regarding the type of application data 
that is to be analyzed. In Figure 9, the user then 
selects the kernel size. In the embodiment of Fig- 

25 ure 10, once the user supplies data regarding what 
type of application the machine is to be used on, 
the user is then prompted to select the kernel 
shape as well as to select the kernel size. In yet 
another alternative embodiment, the computer may 

30 select both the shape and the size based on the 
application data supplied by the user. 

Figure 1 1 shows an example of two different 
kerne! shapes for use on band type data such as is 
found in one-dimensional gel protein separations. 

35 The kernel shape indicated by the dashed line 
labeled 100 is not a good shape to use in this 
situation since it overlaps a portion of the dead 
space 102 which lies between band 104 and band 
106. Since there is no valid background noise in 

40 the dead space 102, the kernel shape 100 will 
distort the background image, thereby creating er- 
rors. The kernel shape 108 is a better shape to use 
for this situation since it includes areas of the band 
104 outside the data band of interest 110 but does 

45 not include any pixels in the dead space 102. 

Figure 12 illustrates a situation where differing 
kernel shapes are useful. The more or less circular 
data spots in Figure 1 2 would be best quantized by 
the use of a circular kernel such as that shown in 

so dashed lines at 112. However, certain types of data 
include vertical strips of noise in the image such as 
is shown at 114. In these situations, it is useful to 
do a two-stage background image generation pro- 
cess. The first stage of this process is to use a 

55 slender vertical kernel which is thinner than the 
thinnest noise streaks in the image. Such a kernel 
is shown at 116 in dashed lines. This kernel shape 
can effectively remove noise strips such as shown 
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at 114. After a background image is generated 
using the kernel shape of 116, the second stage of 
the background image generation process is en- 
tered where the kernel shape changes to that 
shown at 112. Background image generation then 
proceeds operating upon the image generated us- 
ing the kernel 1 12 on the acquired image. 

Referring to Figure 13, there is shown a 
flowchart for the preferred embodiment of a pro- 
cess according to the teachings of the invention. 
The first three stages in the process are symbol- 
ized by blocks 66, 68 and 70. These three stages 
are identical with the first three stages in the pro- 
cess symbolized by the block diagram of Figure 4. 
Likewise, the last two stages, symbolized by blocks 
90 and 92 are identical to the process stages 
symbolized by blocks 90 and 92 in Figure 4. The 
difference between the process symbolized by Fig- 
ure 4 and the process symbolized by Figure 13 
lies in processed stages symbolized by blocks 120 
and 122. 

The process represented by block 120 is a 
series of steps to remove high frequency, large 
amplitude noise from the background image gen- 
erated by the process repre sented by block 70. 
Such high frequency, large amplitude noise typi- 
cally results from pinholes in the film, the penetra- 
tion of gamma rays through the film or other such 
phenomena which cause large spikes in the inten- 
sity values of pixels. The details of this process will 
be given with reference to Figure 14. 

Referring to Figure 14, there is shown a 
flowchart symbolizing the process steps imple- 
mented by block 120 in Figure 13. The process 
represented by Figure 14 essentially generates a 
maximum image from the background image gen- 
erated by block 70 in Figure 13. This is done using 
a smaller kernel than was used to generate the 
background image and by searching throughout 
the kernel to find the maximum pixel value and 
using that value to update the value of the pixel of 
interest within that kernel. This process is repeated 
for all or some subset of all of the pixels in the 
image to generate a maximum image. 

The first step in generating a maximum image 
is symbolized by block 124 representing the pro- 
cess of making a copy of the background image 
generated in the process represented by block 70 
in Figure 13. 

Next, a kernel is selected as symbolized by 
block 126. This kernel should be smaller than the 
kernel used to generate the background image 
and, generally, is very small in that it has an area 
which corresponds to the area of pinhole type 
defects. 

Next, in block 128 the copy image is offset to 
align any selected pixel in each kernel with the 
corresponding pixel of interest in the background 



image. This process is identical to the process 
described with reference to FIGS. 6, 7 and 8 ex- 
cept that a much smaller kernel is used. 

Block 130 represents the process of comparing 

5 each pair of aligned pixels to determine which one 
has the maximum value. This process is also iden- 
tical to the process used in generating the back- 
ground image, but the neighboring pixel in the 
kernel is checked to determine if its value is great- 

w er than the value of the pixel of interest rather than 
less than as in the case of generating the back- 
ground image. 

Block 132 represents the process of updating 
the pixel in the copy image corresponding to the 

75 pixel of interest in the background image for each 
aligned pixel pair where the aligned pixel in the 
copy image has a value which is greater than the 
aligned pixel of interest in the background image. 
This process also corresponds to the background 

20 image generation process described with reference 
to FIGS. 6, 7 and 8 and need not be further 
described here. 

Next, in block 134, the copy image is offset to 
a different location to align another pixel in each 

25 kernel with the pixel of interest in the correspond- 
ing kernel in the background image. 

Then the test of block 136 is performed to 
determine if all the other pixels in the kernel se- 
lected for generation of the maximum image from 

30 the background image have been checked against 
the pixel of interest in each kernel. If all the neigh- 
boring pixels each- kernel have been checked, the 
test of block 136 causes branching to block 138 
where exit to the next step in the process is per- 

35 formed based upon completion of the maximum 
image. The next step in the process would be 
block 122 in Figure 13 in the preferred embodi- 
ment. However, in alternative embodiments, the 
next step in the process would be block 90 in 

40 Figure 13 or some other image processing step. If 
the test of block 136 indicates that not all the pixels 
in the kernel have been checked for a value which 
exceeds the value of the pixel of interest, then a 
branch to block 130 is performed where each pair 

45 of aligned pixels in all the kernels are checked as 
previously described. Steps 130, 132, 134 and 136 
are performed as many times as there are neigh- 
boring pixels to the pixel of interest in each kernel. 
These steps 130, 132 and 134 along with step 136 

so result in the simultaneous processing of the entire 
image. 

Referring to Figure 15, there is shown a 
flowchart of the process represented by block 122 
in Figure 1 3. This process smooths the background 
55 image by averaging all of the pixels in a kernel 
thereby removing high frequency, low amplitude 
noise. The process of Figure 15 can be carried out 
using the background image generated by the pro- 
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cess of block 70 as the starting image in some 
embodiments or upon the image generated by the 
process represented by block 120 in Figure 13 as 
the starting image. That is, alternative embodi- 
ments to the process symbolized by the flowchart 
at Figure 13 are to perform either the process 
represented by block 120 alone or the process 
represented by block 122 alone or both between 
the processes represented by blocks 70 and 90. 
Accordingly, the first stage in the process repre- 
sented by Figure 15 is symbolized by block 140 
and making a copy of the starting image where the 
starting image may be the image generated by the 
process represented by block 70 in Figure 13 or 
the image generated by the process represented 
by block 120. 

Next, in block 142 a kernel is selected. In some 
embodiments, the computer may automatically se- 
lect this kernel, and in other embodiments, the user 
may select a kernel, in either embodiment, the size 
and/or shape may be variable. The size of the 
kernel is generally substantially smaller than the 
kernel used to generate the background image as 
selected in block 68 of Figure 13. 

Block 144 represents the process of offsetting 
the copy image from the starting image to align 
one of the pixels in each kernel having the shape 
and size selected in block 1 42 with the correspond- 
ing pixel of interest in each kernel. This process is 
similar to the process represented by block 128 in 
Figure 14 and the process discussed with refer- 
ence to FIGS. 5-8. 

Next, in block 146 the pair of aligned pixels are 
summed with the sum being used to update the 
pixels in the copy image which are aligned with 
pixels in the starting image. In some embodiments, 
pixels which have no overlying pixel in the copy 
image are summed with a constant. 

In block 148 a process is carried out to offset 
the summed image generated by the process of 
block 146 to align another pixel from each kernel 
with the corresponding pixel of interest in each 
kernel. 

The test of block 150 is to determine if all other 
pixels in each kernel have been aligned with and 
summed with the pixel of interest in each kernel 
with the total being used to update the value of the 
pixel and the summed image which corresponds to 
the pixel of interest in each kernel. In other words, 
steps 144, 146 and 148 are performed a number of 
times equal to the number of pixels in a kernel less 
one. This means that every other pixel in the kernel 
is aligned with the pixel of interest and summed 
therewith. If the test of block 150 determines that 
not all pixels in each kernel have been summed 
with the pixel of interest, branching back to the 
process represented by block 146 occurs. If all 
other pixels in a kernel have been summed, the 



process of block 152 is performed. In this process, 
each pixel in the summed image has its value 
divided by the number of pixels in each kernel. 
This generates a value for each pixel in the 

5 summed image which is the average value for all 
pixels and the kernel. This average value is then 
used to update the value of the pixel in the 
summed image. 

The resultant image is used by block 90 in 

w Figure 13 as the background image which is sub- 
tracted from the acquired image to leave a back- 
ground removed image which is displayed by the 
process of block 92 in Figure 13. 

Block 154 represents the process of repeating 

75 the smoothing or averaging process if desired or 
necessary. If not desired or not necessary, exit to 
block 90 in Figure 13 is performed. 

Referring to Figure 16, there is shown a flow 
chart for a process of generating a percent change 

20 image useful in comparing two data images. Pref- 
erably, this process is carried out on a background 
removed image, but it may also be carried out 
between any two images. The process starts as 
symbolized in block 1 70 by subtracting the value of 

25 a pixel in image 1 from the value of a correspond- 
ing pixel in image 2. Typically, this process would 
be carried by subtracting pixel 1 of line 1 of image 
1 from pixel 1 of line 1 of image 2 and storing the 
difference. However, the order in which the pixel 

30 are process is immaterial as long as corresponding 
pixels, i.e., pixels having the same relative location 
in the image are subtracted. An additional feature 
in some embodiments including the preferred em- 
bodiment is to clip the noise from the percent 

35 change image by implementing a rule limiting the 
allowable differences. The rule is, if the sum of the 
values of the two pixels being compared is less 
than a noise clipping constant (fixed for any ap- 
plication but modifiable by the user), then the dif- 

40 ference is set to 0. This rule has the effect of 
eliminating salt and pepper noise from the percent 
change image which can result when the differ- 
ences between the images at substantially all the 
pixels is small. 

45 Next, in step 172, the difference value is multi- 

plied by a constant. This constant may, in some 
embodiments be fixed for all comparisons, but, in 
the preferred embodiment, the constant is selected 
by the computer based upon the application, but 

50 the user can override the selection and supply a 
new constant. 

Step 174 represents the process of dividing the 
difference between the two pixel values by the 
minimum of the pixel values compared. This step 

55 generates a percentage change number indicating 
how much the intensity or value of the one of the 
pixels varies from the value of the other pixel. 
These percentage numbers vary from 255% to 1 % 
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because the maximum pixel intensity value is 255 
and the minimum pixel intensity value is 1. 

The significance of the constant is that it con- 
trols what percentages changes can be seen in the 
final percent change image and the intensities at 
which the percent change pixels are displayed. 
That is, the constant controls the range of percent- 
age differences which can be seen by stepping up 
the percentage change numbers to larger numbers. 
However, the maximum intensity value which can 
be displayed is 255. Therefore, selection of larger 
constants can lead to clipping since the resulting 
percentage change numbers after multiplication 
can exceed 255. In the preferred embodiment, the 
constant ranges from +1 to +256, but in other 
embodiments, any number between 0 and any 
positive number could be used including fractional 
numbers. 

In step 176, the result from step 174 is added 
to another constant to set the 0% change number 
equal to some reference intensity value. In the 
preferred embodiment, intensity values range from 

1 to 255, and the constant used in step 176 is 127 
such that the 0% change falls in the middle of the 
gray scale. 

In step 178, the results from step 176 are 
clipped between 0 and 255 for purposes of using 
the results on a video display. The result is stored 
in a percent change image file or framestore. 

Step 180 represents the process of repeating 
steps 170 through 178 for all pixels in both images 
to complete generation of the percent change im- 
age. The image may then be displayed for inspec- 
tion and analysis. 

Referring to Figures 17(a) through 17(b), there 
are shown a plurality of images which together 
comprise the components of a quad display and 
the quad display itself. The purpose of a quad 
display is to facilitate visual comparisons and ana- 
lysis of data bearing images. The components of a 
quad display are the two compared images used to 
generate the percent change image, the percent 
change image itself, and a fourth image which is 
called the difference image. This difference image 
is at each pixel the difference between the two 
corresponding pixels in images 1 and 2, divided by 

2 and added to 127. The percent change image is 
the image generated by the process of Figure 16. 
The term "corresponding pixels" for the description 
of the difference image means the same thing as 
that term as used for the percent change image. 

If the display hardware is large enough to dis- 
play four complete images of the size of image 1, 
then all four images are simultaneously displayed 
as arranged in Figure 17(e). If however, the display 
apparatus can display only one image having the 
number of pixels in image 1, then several alter- 
native embodiments are possible. First, each image 



may be sampled to develop of subset of pixel for 
that image. Such sampling can include selecting 
every other pixel on every other line such that one- 
fourth of the total number of pixels remain to be 

5 displayed. In another alternative embodiment, a 
selection of one-quarter of each image is made, 
and that quarter of an image is displayed in the 
corresponding portion of the quad display of Figure 
17(e). The quarter of each image selected can be 

10 selected by the user or by the computer based 
upon the application or can be set to a default 
selection by the computer and modifiable by the 
user. The possibilities for which one-quarter to se- 
lect are numerous and include one quadrant of 

75 each image, a horizontal strip amounting to one- 
fourth of the pixels or a vertical strip amounting to 
one-fourth of the pixels. 

In the preferred embodiment, selects which 
quarter of each image to be displayed by manipu- 

20 lation of a "linked" cursor in a scout image. The 
scout image is, in the referred embodiment, a 2 to 
1 minification or subset of image 2. This minifica- 
tion is performed by selecting every other pixel of 
every other line and displaying the result as the 

25 scout image in the lower left quadrant of the dis- 
play. The locked cursor is a fixed cursor encom- 
passing one-fourth of the total area of the scout 
image. The user manipulates the position of this 
cursor by manipulation of a mouse, track ball or 

30 light pen etc. Also, the position of the cursor can 
be positioned by default in one of the four quad- 
rants of the scout image and this position can then 
be modified by the user. As the user moves the 
cursor in the scout image, a corresponding cursor 

35 in each of the other four images moves synchro- 
nously to encompass the pixels corresponding to 
the pixels encompassed by the cursor in the scout 
image. When the user, selects a position for the 
cursor in the scout image as the final position, the 

40 corresponding pixels in the other three images are 
selected and displayed in the corresponding quad- 
rants of the quad display shown in Figure 17(e). 
Simultaneously, all the pixels in the difference im- 
age corresponding to the selected pixels in images 

45 1,2 and the percent change image are selected 
and displayed in the lower left quadrant of the quad 
display. 

In other embodiments, the quad display may 
be arranged differently. An example of such an 

50 alternative arrangement is as shown in Figure 18. 
Any combination of hardware and software to im- 
plement this process and cursor manipulation will 
suffice for purposes of practicing the invention. The 
preferred embodiment of computer code which 

55 when combined with the hardware illustrated in 
Figure 3 will implement the teachings of the inven- 
tion is given in Appendix A. 

Referring again to Figure 17(e) there is shown 
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the positions for four measurement cursor locations 
covering four sets of corresponding pixels. These 
cursor locations are shown at 182, 184, 186 and 
188 in exemplary rectangular shape. The shape 
and size of the cursors can be selected by default 
by the computer and be modifiable by the user or 
can be selected outright by the user. 

After the position, shape and size of the cursor 
is established, the computer calculates some quan- 
tity related to the values of the pixels inside the 
cursor. Examples of what these quantities can be 
are: 1) absorption meaning the sum of all the pixel 
values within the cursor in preselected units, which 
can be optical density, counts per minute, etc. for 
images 1 and 2 only; 2) the average value of all 
pixels in each cursor location for images 1 and 2; 
3) square millimeters of optical density meaning 
absorption in optical density units divided by the 
number of pixels per square millimeter. Once these 
values are calculated for images 1 and 2, the 
values for the corresponding sets of pixels in the 
cursor locations in the percent change and dif- 
ference images are automatically determined. That 
is, for the percent change image, the value re- 
turned for the cursor location is calculated accord- 
ing to the algorithm specified in Figure 16 as 
modified by omission of multiplication by the con- 
stant and addition of the second constant. That is, 
the number returned for the cursor location in the 
percent change image is the value determined for 
the cursor in image 1 minus the value for the 
cursor in image 2 divided by the minimum value 
between these two numbers. 

Likewise, the value returned for the cursor in 
the difference image is the value of the difference 
between the values returned for the cursor loca- 
tions in images 1 and 2 divided by 2. 

Figure 19 clarifies how the cursor manipulated 
by the user in the 2 to 1 minified scout image 
(subset of image 2) corresponds to one-quarter of 
the pixels in the full size image 2 at the same 
relative location in the image. 

Referring to Figure 20, there is shown a dia- 
gram illustrating the process of alignment of im- 
ages 1 and 2 which is necessary for the computa- 
tion of values for the pixels in the cursors in the 
quad display. Figure 20(a) represents the camera 
input video data that defines image 2. Figure 20(b) 
illustrates an already acquired image 1 stored in a 
frame buffer. The corresponding pixels from these 
two images are combined according to the al- 
gorithm specified in Figure 20(d) to generate the 
image of Figure 20(c) on the display. The user then 
manipulates image 2 under the camera until the 
displayed image calculated per the equation of 
Figure 20(d) shows minimal difference between 
image 1 and image 2. When this condition exists, 
the user so indicates, and the pixels of the image 



of Figure 20(a) are captured in a frame buffer as 
the final image 2 for use in the processing de- 
scribed above to return the values for the selected 
cursor locations in the quad display shown in Fig- 

5 ure 17. 

Although the invention has been described in 
terms of the preferred and alternative embodiments 
disclosed herein, those skilled in the art will appre- 
ciate numerous modifications which can be made 

w without departing from the spirit and scope of the 
invention. All such modifications are intended to be 
included within the scope of the claims appended 
hereto. 

75 

Claims 

1 . An apparatus for removing background noise 
from an image, comprising: 

20 means for digitizing an image into a plurality of 
pixels defining a first image, each pixel having a 
value, and for making and storing a copy of said 
first image as a copy image; 
means for generating a background image from 

25 said copy image by simultaneous processing on 
each said pixel, said simultaneous processing com- 
prising, for each selected pixel in said copy image, 
comparing the value of said selected pixel with the 
value of another pixel in a group of pixels adjacent 

30 to said selected pixels and replacing the value at 
said selected pixel with the value of said other pixel 
if said other pixel's value is less than the value of 
said selected pixel, and repeating said comparison 
and replacement process for at least selected ones 

35 of said other pixels in said group; and 

means for subtracting said background image from 
said first image to generate a background-removed 
image. 

2. The apparatus of claim 1 further comprising 
40 means for displaying said background-removed im- 
age. 

3. The apparatus of claim 1 wherein said 
means for generating includes means for selecting 
said other pixels in said group so that said other 

45 pixels define a predetermined shape. 

4. The apparatus of claim 3 wherein said 
means for selecting includes means for selecting 
said other pixels so as to define a group with a 
predetermined shape. 

50 5. The apparatus of claim 1 wherein said 

means for digitizing creates a digital image that 
displays data in spatial patterns having predeter- 
mined size and shape characteristics, and wherein 
said means for generating includes means for se- 

55 lecting said other pixels in said group includes 
means to select said other pixels so as to define a 
group having predetermined size and shape char- 
acteristics relative to said size and shape char- 
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acteristics of said data. 

6. The apparatus of claim 5 wherein said 
means for selecting selects said other pixels so as 
to define said group with a shape which substan- 
tially matches the shape of a selected data shape. 

7. The apparatus of claim 6 wherein said 
means for selecting includes means for selecting a 
variable size for said group such that said group 
size can be substantially matched to the size of a 
selected area of data. 

8. An apparatus for removing background noise 
from a digital image which displays data spatially 
as a plurality of pixels in a raster display, each 
pixel having a value, comprising: 

means for digitizing an image containing said data 
and background noise to generate a first image; 
means for removing said background noise by re- 
peated simultaneous processing of selected groups 
of pixel pairs between a copy of said first image 
hereafter called a copy image and said first image 
to generate a background image from said copy 
image; and 

means for subtracting said background image from 
said first image. 

9. The apparatus of claim 1 wherein said se- 
lected group of pixels includes a kernel of pixels in 
said copy image which define a shape and size in 
said copy image which have predetermined rela- 
tionships to the shape and size of selected data in 
said first image. 

10. The apparatus of claim 8 wherein said 
means for removing includes means to select said 
pixels in said kernel to define a predetermined 
shape relative to the shape of data to be analyzed 
in said first image and further comprising means to 
select a variable sized group of said pixels in said 
kernel so as to maintain said predetermined shape 
but sized according to an input signal. 

11. The apparatus of claim 8 wherein said 
means for removing includes means to select said 
pixels in said kernel so as to have a shape which 
matches the typical shape of data to be analyzed 
in said first image and so as to have a size which 
is larger than the largest sized group of data pixels 
to be analyzed. 

12. The apparatus of claim 9 further comprising 
means to remove high-frequency, large-amplitude 
noise from said background image to generate a 
smoothed image. 

13. The apparatus of claim 12 further compris- 
ing means for removing high-frequency, low-am- 
plitude noise from said smoothed image to gen- 
erate a filtered background image. 

14. The apparatus of claim 8 further comprising 
means for removing high-frequency, low-amplitude 
noise from said background image from said first 
image. 

15. The apparatus of claim 8 further comprising 



means for removing high-frequency, large ampli- 
tude noise from said background image before 
subtracting said background image from said first 
image. 

s 16. An automated noise removal system in a 

system to graphically display data spatially as 
areas of varying intensity on a video display to 
preserve the intensity of data-bearing data dis- 
played in spatial features of greater than a pre- 

10 determined size while removing background noise 
also displayed graphically and spatially on a video 
display comprising: 

a uniform illumination light box on which a trans- 
parent or translucent medium is placed, said me- 
75 dium having spatially depicted thereon the data to 
be analyzed; 

a video camera positioned adjacent said light box 
so that there is an optical pathway therebetween, 
said camera providing a video output comprising 

20 an analog video signal depicting said data on said 
medium in terms of the spatial patterns of light 
intensity of light from said light box which passes 
through said medium with said light intensity being 
modulated by the spatial patterns of data and back- 

25 ground noise depicted on said medium; 

converter means coupled to receive said analog 
video signal for converting said analog video signal 
to a stream of digital data and for storing said 
digital data in a memory to define a digital first 

30 image comprised of a plurality of pixels arranged in 
rows and columns spatially displaying said data 
along with background noise; 
background removal means coupled to said con- 
verter means for simultaneously comparing the in- 

35 tensity value at each pixel in said first image with a 
corresponding, predetermined intensity value of a 
pixel in an offset image, where said offset image is 
a copy of said first image but offset from said first 
image by a predetermined number of rows and 

40 columns, and at each pixel location in said offset 
image establishing the intensity value as the inten- 
sity value which is smallest as between the pair of 
compared pixel intensity values which correspond 
to said pixel location in said offset image and for 

45 repeating the offsetting and comparison and writing 
of intensity values for the offset image for each of a 
plurality of different offset values defining a neigh- 
borhood shape for a predetermined group of pixels 
in said first image thereby establishing said offset 

50 image as a background image where each selected 
pixel's intensity value corresponds to the minimum 
intensity value for all pixels in a group of pixels in 
said first image which are adjacent to the pixel 
which corresponds to said selected pixel when 

55 there is a zero row-and-column offset, said group 
of pixels defining a shape having the shape of said 
neighborhood; 

means for subtracting said background image from 
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said first image on a pixel-by-pixel basis to gen- 
erate a background-removed image; and 
means for converting said background-removed im- 
age to a video signal and for displaying said video 
signal on a video display. 

17. An operator-interactive automated noise 
background removal system to preserve the am- 
plitude of data-bearing spatial features greater than 
a prescribed size while removing background and 
high-frequency noise from an image of biological 
data acquired from an autoradiograph, elec- 
trophoresis gel, fluorescence gel, photographic film 
or other media, said system comprising: 

a means for displaying an image containing data 
shown by spatial patterns of varying intensity; 
image conversion means for generating video out- 
put data from said image and positioned adjacent 
said means for displaying said image so that there 
is an optical pathway therebetween, said image 
conversion means for converting said image to 
video output data at a video output; 
background detector means connected via an 
analog-to-digital converter pathway for receiving 
video output data and converting said video data to 
digital data, said background detector means pro- 
cessing said digital data so as to detect valid 
background values within the image of data; 
computer means connected to said background 
detector means and having an interactive mode for 
the operator to specify upper size bounds of valid 
spatial features, said background detector deter- 
mining the background neighborhood used in sur- 
rounding spatial features of the data; 
monitor means connected to said background de- 
tector means and responsive to said video output 
for displaying a television image of said 
background-removed data; 

18. The system of claim 17, wherein said back- 
ground detector system means comprises means 
for converting said optical image data to digital 
image data and for causing said optical image data 
to be, upon receipt of a reverse command, con- 
verted to reverse video digital data after said con- 
version to digital image data. 

19. The system of claim 18, wherein said 
means for converting comprises an analog to digi- 
tal converter. 

20. The system of claim 17, wherein said back- 
ground detector means comprises first, second and 
third discriminators for detecting when said re- 
ceived digital data is valid signal comprising spatial 
features in the biological data. 

21. The system of claim 20, wherein said first 
discriminator means comprises means for generat- 
ing a background image which is subtracted from 
said image of biological data. 

22. The system of claim 21, wherein said back- 
ground image comprises digital image data which 



consists of a rectangular array of pixels each of 
whose intensity is the minimum within a neighbor- 
hood of each pixel in said biological image data. 

23. The system of claim 22, wherein said array 
5 of pixels comprises digital data having digital inten- 
sity values which are shades of gray when dis- 
played on said monitor means. 

24. The system of claim 20, wherein said sec- 
ond discriminator means comprises means for de- 

w termination of a neighborhood size for use in dis- 
criminating out noise which enables measurement 
of the amplitude of features and objects in said 
image of biological data where the base plateau 
beneath the features and objects is nonuniform. 

75 25. The system of claim 22, wherein said third 

discriminator means comprises means for smooth- 
ing of said background image by removing or 
damping high-frequency, low-amplitude noise in 
said background image of biological data. 

20 26. The system of claim 22, wherein said 

fourth discriminator means comprises means for 
clipping of said background image so as to clip 
noise which is high frequency as compared with 
the frequency of the signal for said data-bearing 

25 spatial features which is also high amplitude rela- 
tive to the amplitude of the signal in said image of 
biological data where data-bearing spatial features 
are not present. 

27. The system of claim 17, wherein said back- 
30 ground detector includes means for subtraction of 

said background image from said image of biologi- 
cal data which preserves data-bearing spatial fea- 
tures and enables measurement of the amplitude of 
the signal comprising data-bearing spatial features 
35 within said image of biological data. 

28. The system of claim 24, wherein said sec- 
ond discriminator means further comprises means 
for selecting neighborhood size through user inter- 
action to separate data-bearing spatial features in 

40 said images of biological data so as to pick a 
neighborhood size and shape suited to enable 
measurement of quantity of biological materials re- 
presented by multiple data-bearing spatial features 
in close proximity. 

45 29. The system of claim 17, wherein said com- 

puter means is connected to said image conversion 
means for receiving video output data therefrom, 
said computer means comprising a video interface 
for converting said video output data to digital 

50 signals and storing the digital signals. 

30. The system of claim 17, wherein said com- 
puter means comprises an interface circuit and a 
computer connected thereto, said interface circuit 
converting digital signals received form said com- 

55 puter to analog signals and sending said analog 
signals to said monitor means, in response to 
which said monitor means displays said 
background-removed images of valid spatial fea- 

14 
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tures. 

31. An apparatus for removal of noise from a 
digital image displaying data spatially as a plurality 
of pixels on a raster-scanned video display com- 
prising: 5 
a background removal circuit for making a copy of 
said digital image and comparing a selected pixel 

in said digital image to a neighborhood of adjacent 
pixels using said copy and finding the minimum 
pixel value in said neighborhood and setting a pixel 10 
in said copy cor responding to said selected pixel 
to said minimum pixel value and for doing this 
process simultaneously for a predetermined num- 
ber of pixels in said digital image to convert said 
copy to a background image; 75 
a subtracter for subtracting said background image 
from said digital image to remove noise from said 
digital image. 

32. The apparatus of claim 31 further compris- 
ing a circuit for receiving data regarding the size 20 
and shape of said neighborhood best suited for 
optimal noise removal and for causing said back- 
ground removal circuit to use such a neighborhood 

in noise removal. 

33. The apparatus of claim 32 further compris- 25 
ing a high-frequency filter in said background re- 
moval circuit for removing high-frequency noise 
from said background image before subtracting 

said background image from said digital image. 

34. A method of removing noise from a data- 30 
bearing image spatially displaying data, comprising 

the steps of: 

simultaneously comparing each pixel in said image 
to a neighborhood of adjacent pixels and setting 
said each pixel to the minimum value found in said 35 
neighborhood to generate a background image; 
subtracting said background image from said data- 
bearing image. 

35. The method of claim 34 further comprising 

the step of selecting the size and shape of said 40 
neighborhood for optimum removal of noise from 
said data-bearing image given the size and shape 
of said spatial displays of data. 

36. The method of claim 35 further comprising 

the step of removing or damping high-frequency 45 
noise in said background image before subtracting 
said background image from said data-bearing im- 
age. 
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